Focused Retrieval and Evaluation
Traditional document retrieval has shown to be a competitive approach in XML element retrieval, which is counter-intuitive since the element retrieval task requests all and only relevant document parts to be retrieved. This paper conducts a comparative analysis of document and element retrieval, highlights the relative strengths and weaknesses of both approaches, and explains the relative effectiveness of document retrieval approaches at element retrieval tasks.
Third workshop on exploiting semantic annotations in information retrieval %28ESAIR%29 CIKM 2010 workshop.
There is an increasing amount of structure on the Web as a result of modern Web languages, user tagging and annotation, and emerging robust NLP tools. These meaningful, semantic, annotations hold the promise to significantly enhance information access, by enhancing the depth of analysis of today’s systems. Currently, we have only started exploring the possibilities and only begin to understand how these valuable semantic cues can be put to fruitful use. Unleashing the potential of semantic annotations requires us to think outside the box, by combining the insights of natural language processing (NLP) to go beyond bags of words, the insights of databases (DB) to use structure efficiently even when aggregating over millions of records, the insights of information retrieval (IR) in effective goal-directed search and evaluation, and the insights of knowledge management (KM) to get grips on the greater whole. The Workshop aims to bring together researchers from these different disciplines and work together on one of the greatest challenges in the years to come. The desired result of the workshop will be concrete insight into the potential of semantic annotations, and in concrete steps to take this research forward; synchronize related research happening in NLP, DB, IR, and KM, in ways that combine the strengths of each discipline; and have a lively, interactive workshop were everyone contributes and that inspires attendees to think “outside the box.”
Entity ranking using Wikipedia as a pivot.
In this paper we investigate the task of Entity Ranking on the Web. Searchers looking for entities are arguably better served by presenting a ranked list of entities directly, rather than a list of web pages with relevant but also potentially redundant information about these entities. Since entities are represented by their web homepages, a naive approach to entity ranking is to use standard text retrieval. Our experimental results clearly demonstrate that text retrieval is effective at finding relevant pages, but performs poorly at finding entities. Our proposal is to use Wikipedia as a pivot for finding entities on the Web, allowing us to reduce the hard web entity ranking problem to easier problem of Wikipedia entity ranking. Wikipedia allows us to properly identify entities and some of their characteristics, and Wikipedia’s elaborate category structure allows us to get a handle on the entity’s type. Our main
A Search Log%2DBased Approach to Evaluation.
Abstract. Anyone offering content in a digital library is naturally interested in assessing its performance: how well does my system meet the users ’ information needs? Standard evaluation benchmarks have been developed in information retrieval that can be used to test retrieval effectiveness. However, these generic benchmarks focus on a single document genre, language, media-type, and searcher stereotype that is radically different from the unique content and user community of a particular digital library. This paper proposes to derive a domain-specific test collection from readily available interaction data in search logs files that captures the domain-specificity of digital libraries. We use an archival institution’s complete search log that spans over multiple years, and derive a large-scale test collection. We use this resulting domain specific test collection to evaluate a range of retrieval models. Finally, we manually derive a set of topics judged by human experts—based on a set of e-mail reference questions and responses from archivists—and use this for validation. Our main finding is that we can derive a massive, reliable and domain-specific test collection from search log files. 1
How Different Are Language Models andWord Clouds%3F

An Empirical Study of Query Specificity.
Abstract. We analyse the statistical behavior of query-associated quantities in query-logs, namely, the sum and mean of IDF of query terms, otherwise known as query specificity and query mean specificity. We narrow down the possibilities for modeling their distributions to gamma, log-normal, or log-logistic, depending on query length and on whether the sum or the mean is considered. The results have applications in query performance prediction and artificial query generation. 1
The importance of anchor text for ad hoc search revisited.

The impact of collection size on relevance and diversity.

The impact of collection size on relevance and diversity.

Linking wikipedia to the web.
Contact: 9862005729 Quality of higher education is a multi-dimensional concept. It lies in effectiveness of transmitting knowledge and skill; the authenticity, content, coverage and depth of information; availability of reading/teaching materials; help in removing obstacles to learning; applicability of knowledge in solving the real life problems; fruitfulness of knowledge in personal and social domains; convergence of content and variety of knowledge over space (countries and regions) and different sections of the people; cost-effectiveness and administrative efficiency. Information technology has progressed very fast in the last three decades; it has produced equipments at affordable cost and it has now made their wider application feasible. This technology has made search, gathering, dissemination, storing, retrieval, transmission and reception of knowledge easier, cheaper and faster. Side by side, a vast virtual library vying with the library in prints has emerged and continues growing rapidly. One may hold that the elibraries are the libraries of tomorrow when the libraries in prints will be the antiques or the archival objects of the past. This paper discusses in details how information technology can be applied to enhance the quality of higher education at affordable cost. It also discusses the major obstacles to optimal utilization of information technology and measures to remove them.
Linking wikipedia to the web.
Contact: 9862005729 Quality of higher education is a multi-dimensional concept. It lies in effectiveness of transmitting knowledge and skill; the authenticity, content, coverage and depth of information; availability of reading/teaching materials; help in removing obstacles to learning; applicability of knowledge in solving the real life problems; fruitfulness of knowledge in personal and social domains; convergence of content and variety of knowledge over space (countries and regions) and different sections of the people; cost-effectiveness and administrative efficiency. Information technology has progressed very fast in the last three decades; it has produced equipments at affordable cost and it has now made their wider application feasible. This technology has made search, gathering, dissemination, storing, retrieval, transmission and reception of knowledge easier, cheaper and faster. Side by side, a vast virtual library vying with the library in prints has emerged and continues growing rapidly. One may hold that the elibraries are the libraries of tomorrow when the libraries in prints will be the antiques or the archival objects of the past. This paper discusses in details how information technology can be applied to enhance the quality of higher education at affordable cost. It also discusses the major obstacles to optimal utilization of information technology and measures to remove them.
Current research in focused retrieval and result aggregation.
Abstract. Both focused retrieval and result aggregation provide the user with answers to their information needs, rather than just pointers to whole documents. Focused retrieval identifies not only relevant documents but also which parts of those documents are relevant, thus reducing the time it takes the user to navigate in a document. Result aggregation is used when the best way to fulfil the user’s need is to draw from many different information sources (different collections, documents, or parts of the same document), and to aggregate these into a single result, thus reducing the time it takes the user to fulfil their information need. This special issue includes seven papers showing the breadth and depth of the current state of research in these two branches of information retrieval. They include descriptions of live result aggregation systems, and experimental focussed retrieval systems including sentence retrieval, question answering, and entity ranking; as well as metrics for passage retrieval. To introduce this special issue, we provide an overview of the work presented in these papers. 1.
Current research in focused retrieval and result aggregation.
Abstract. Both focused retrieval and result aggregation provide the user with answers to their information needs, rather than just pointers to whole documents. Focused retrieval identifies not only relevant documents but also which parts of those documents are relevant, thus reducing the time it takes the user to navigate in a document. Result aggregation is used when the best way to fulfil the user’s need is to draw from many different information sources (different collections, documents, or parts of the same document), and to aggregate these into a single result, thus reducing the time it takes the user to fulfil their information need. This special issue includes seven papers showing the breadth and depth of the current state of research in these two branches of information retrieval. They include descriptions of live result aggregation systems, and experimental focussed retrieval systems including sentence retrieval, question answering, and entity ranking; as well as metrics for passage retrieval. To introduce this special issue, we provide an overview of the work presented in these papers. 1.
Report on INEX 2009.
INEX investigates focused retrieval from structured documents by providing large test collections of structured documents, uniform evaluation measures, and a forum for organizations to compare their results. This paper reports on the INEX 2009 evaluation campaign, which consisted of a wide range of tracks: Ad hoc, Book, Efficiency, Entity Ranking, Interactive, QA, Link the Wiki, and XML Mining. INEX in running entirely on volunteer effort by the IR research community: anyone with an idea and some time to spend, can have a major impact! 1
Advances in Focused Retrieval

A signal%2Dto%2Dnoise approach to score normalization.
Score normalization is indispensable in distributed retrieval and fusion or meta-search where merging of result-lists is required. Distributional approaches to score normalization with reference to relevance, such as binary mixture models like the normal-exponential, suffer from lack of universality and troublesome parameter estimation especially under sparse relevance. We develop a new approach which tackles both problems by using aggregate score distributions without reference to relevance, and is suitable for uncooperative engines. The method is based on the assumption that scores produced by engines consist of a signal and a noise component which can both be approximated by submitting well-defined sets of artificial queries to each engine. We evaluate in a standard distributed retrieval testbed and show that the signal-to-noise approach yields better results than other distributional methods. As a significant by-product, we investigate query-length distributions. normalized frequency
Evaluation in Context.

Searching Archival Finding Aids Retrieval in Original Order%3F
Abstract. Archival principles as Provenance (keeping material from the same creator together) and its corollary Original Order (keeping the order of creation intact) could help improve access to the archival materials. We investigate the importance of relevance ranking and ‘Original Order ’ when searching finding aids in EAD using XML Retrieval. Our experiment shows that relevance ranking is of paramount importance, although Original Order may help the retrieval of the first few results because these tend to cluster within the original order. 1
Score Distributions in Information Retrieval.
Abstract. We review the history of modeling score distributions, focusing on the mixture of normal-exponential by investigating the theoretical as well as the empirical evidence supporting its use. We discuss previously suggested conditions which valid binary mixture models should satisfy, such as the Recall-Fallout Convexity Hypothesis, and formulate two new hypotheses considering the component distributions under some limiting conditions of parameter values. From all the mixtures suggested in the past, the current theoretical argument points to the two gamma as the most-likely universal model, with the normal-exponential being a usable approximation. Beyond the theoretical contribution, we provide new experimental evidence showing vector space or geometric models, and BM25, as being “friendly ” to the normal-exponential, and that the non-convexity problem that the mixture possesses is practically not severe. 1
What%27s in a Link%3F From Document Importance to Topical Relevance.
Abstract. Web information retrieval is best known for its use of the Web’s link structure as a source of evidence. Global link evidence is by nature query-independent, and is therefore no direct indicator of the topical relevance of a document for a given search request. As a result, link information is usually considered to be useful to identify the ‘importance ’ of documents. Local link evidence, in contrast, is query-dependent and could in principle be related to the topical relevance. We analyse the link evidence in Wikipedia using a large set of ad hoc retrieval topics and relevance judgements to investigate the relation between link evidence and topical relevance. 1
Analysis of the INEX 2009 Ad Hoc Track Results.

Focused Search in Books and Wikipedia Categories%2C Links and Relevance Feedback.

Overview of the INEX 2009 Ad Hoc Track.
Abstract. This paper gives an overview of the INEX 2009 Ad Hoc Track. The main goals of the Ad Hoc Track were three-fold. The first goal was to investigate the impact of the collection scale and markup, by using a new collection that is again based on a the Wikipedia but is over 4 times larger, with longer articles and additional semantic annotations. For this reason the Ad Hoc track tasks stayed unchanged, and the Thorough Task of INEX 2002–2006 returns. The second goal was to study the impact of more verbose queries on retrieval effectiveness, by using the available markup as structural constraints—now using both the Wikipedia’s layout-based markup, as well as the enriched semantic markup—and by the use of phrases. The third goal was to compare different result granularities by allowing systems to retrieve XML elements, ranges of XML elements, or arbitrary passages of text. This investigates the value of the internal document structure (as provided by the XML mark-up) for retrieving relevant information. The INEX 2009 Ad Hoc
Where to stop reading a ranked list%3F threshold optimization using truncated score distributions.
Ranked retrieval has a particular disadvantage in comparison with traditional Boolean retrieval: there is no clear cut-off point where to stop consulting results. This is a serious problem in some setups. We investigate and further develop methods to select the rank cutoff value which optimizes a given effectiveness measure. Assuming no other input than a system’s output for a query—document scores and their distribution—the task is essentially a score-distributional threshold optimization problem. The recent trend in modeling score distributions is to use a normal-exponential mixture: normal for relevant, and exponential for non-relevant document scores. We discuss the two main theoretical problems with the current model, support incompatibility and non-convexity, and develop new models that address them. The main contributions of the paper are two truncated normal-exponential models, varying in the way the outtruncated score ranges are handled. We conduct a range of experiments


Who said what to whom%3F capturing the structure of debates.
Transcipts of meetings are a document genre characterized by a complex narrative structure. The essence is not only what is said, but also by who and to whom. This paper investigates whether we can use semantic annotions like the speaker in order to capture this debate structure, and as well as the related content of debate. The structure is visualized in a graph, while the content is condensed into word clouds, that are created using a parsimonious language model. Evaluation shows that both tools adequately capture the structure and content of the debate at an aggregated level. Categories and Subject Descriptors:
Focused Search in Digital Archives.
Abstract. We present a system description for an archival information system with three different approaches to gain online access to digital archives created in the metadata standard Encoded Archival Description (EAD). We show that an aggregation-based system can be developed on archival data using XML Information Retrieval (XML IR). We describe the different stages and components, such as the indexing of the digital finding aids in an XML database, the subsequent querying and retrieval of information from that database, and the eventual delivery of that information to the users in a contextual interface. 1
Is Wikipedia link structure different%3F
In this paper, we investigate the difference between Wikipedia and Web link structure with respect to their value as indicators of the relevance of a page for a given topic of request. Our experimental evidence is from two IR test-collections: the.GOV collection used at the TREC Web tracks and the Wikipedia XML Corpus used at INEX. We first perform a comparative analysis of Wikipedia and.GOV link structure and then investigate the value of link evidence for improving search on Wikipedia and on the.GOV domain. Our main findings are: First, Wikipedia link structure is similar to the Web, but more densely linked. Second, Wikipedia’s outlinks behave similar to inlinks and both are good indicators of relevance, whereas on the Web the inlinks are more important. Third, when incorporating link evidence in the retrieval model, for Wikipedia the global link evidence fails and we have to take the local context into account.

Indexing Units.
COWLES FOUNDATION DISCUSSION PAPER NO. 1179

Report on INEX 2008.
INEX investigates focused retrieval from structured documents by providing large test collections of structured documents, uniform evaluation measures, and a forum for organizations to compare their results. This paper reports on the INEX 2008 evaluation campaign, which consisted of a wide range of tracks: Ad hoc, Book, Efficiency, Entity Ranking, Interactive, QA, Link the Wiki, and XML Mining. 1
Focused Access to XML Documents

Report on the SIGIR 2009 workshop on the future of IR evaluation.

Focused Access to XML Documents

Access to Archival Finding Aids Context Matters.
Abstract. We detail the design of a search engine for archival finding aids based on an XML database system. The resulting system shows results—which can vary in granularity from individual archival items to the whole fonds—within the context of the archive. The presentation preserves the archival structure by providing important contextual information, and all individual results can be “clicked”, warping the user to the full finding aid with the selected part in focus. 1
The Importance of Link Evidence in Wikipedia.
Abstract. Wikipedia is one of the most popular information sources on the Web. The free encyclopedia is densely linked. The link structure in Wikipedia differs from the Web at large: internal links in Wikipedia are typically based on words naturally occurring in a page, and link to another semantically related entry. Our main aim is to find out if Wikipedia’s link structure can be exploited to improve ad hoc information retrieval. We first analyse the relation between Wikipedia links and the relevance of pages. We then experiment with use of link evidence in the focused retrieval of Wikipedia content, based on the test collection of INEX 2006. Our main findings are: First, our analysis of the link structure reveals that the Wikipedia link structure is a (possibly weak) indicator of relevance. Second, our experiments on INEX ad hoc retrieval tasks reveal that if the link evidence is made sensitive to the local context we see a significant improvement of retrieval effectiveness. Hence, in contrast with earlier TREC experiments using crawled Web data, we have shown that Wikipedia’s link structure can help improve the effectiveness of ad hoc retrieval. 1
Access to archival material in context.
Archival finding aids are long and complexly structured documents describing archival material—the paper trails of the lives of corporate bodies, persons, and families. Currently, finding aids are encoded in XML using the standard Encoded Archival Description (EAD) and made available to the public on web-sites of archival institutions. But how to provide access to such long and complexly structured documents? On the one hand, users tend to look for specific archival material that may be deeply nested inside the archive. On the other hand, interpreting the meaning of an item is crucially dependent on its context. Using insights from the field of XML retrieval—a subfield of information retrieval that has recently attracted a lot of attention, mainly through the annual evaluation effort in INEX—we developed three different systems for searching in collections of digital finding aids corresponding to three fundamental choices about archival access. The first system provides access to the fonds or archive as a whole; the second system provides direct access to individual archival material at any level of description; the third system retrieves archival material while preserving the original context. This paper reports on the results of an extensive user study with the three systems. Our main finding is that test persons have a preference for the third system that retrieves archival material in their original context, with test persons indicating that the system assisted them in assessing relevancy, navigation and direct access to relevant parts of the finding aids.
Overview of the INEX 2008 Ad Hoc Track.
Abstract. This paper gives an overview of the INEX 2008 Ad Hoc Track. The main goals of the Ad Hoc Track were two-fold. The first goal was to investigate the value of the internal document structure (as provided by the XML mark-up) for retrieving relevant information. This is a continuation of INEX 2007 and, for this reason, the retrieval results are liberalized to arbitrary passages and measures were chosen to fairly compare systems retrieving elements, ranges of elements, and arbitrary passages. The second goal was to compare focused retrieval to article retrieval more directly than in earlier years. For this reason, standard document retrieval rankings have been derived from all runs, and evaluated with standard measures. In addition, a set of queries targeting Wikipedia have been derived from a proxy log, and the runs are also evaluated against the clicked Wikipedia pages. The INEX 2008 Ad Hoc Track featured three tasks: For the Focused Task a ranked-list of nonoverlapping results (elements or passages) was needed. For the Relevant in Context Task non-overlapping results (elements or passages) were returned grouped by the article from which they came. For the Best in Context Task a single starting point (element start tag or passage start) for each article was needed. We discuss the results for the three tasks, and examine the relative effectiveness of element and passage retrieval. This is examined in the context of content only (CO, or Keyword) search as well as content and structure (CAS, or structured) search. Finally, we look at the ability of focused retrieval techniques to rank articles, using standard document retrieval techniques, both against the judged topics as well as against queries and clicks from a proxy log. 1
The Impact of Document Level Ranking on Focused Retrieval.
Abstract. Document retrieval techniques have proven to be competitive methods in the evaluation of focused retrieval. Although focused approaches such as XML element retrieval and passage retrieval allow for locating the relevant text within a document, using the larger context of the whole document often leads to superior document level ranking. In this paper we investigate the impact of using the document retrieval ranking in two collections used in the INEX 2008 Ad hoc and Book Tracks; the relatively short documents of the Wikipedia collection and the much longer books in the Book Track collection. We experiment with several methods of combining document and element retrieval approaches. Our findings are that 1) we can get the best of both worlds and improve upon both individual retrieval strategies by retaining the document ranking of the document retrieval approach and replacing the documents by the retrieved elements of the element retrieval approach, and 2) using document level ranking has a positive impact on focused retrieval in Wikipedia, but has more impact on the much longer books in the Book Track collection. 1
Finding Entities in Wikipedia Using Links and Categories.

A Content%2DBased Link Detection Approach Using the Vector Space Model.
Abstract. Link detection can be seen as a special application of Focused Retrieval. This paper presents a content-based link detection approach using the Vector Space Model. We present our results, and conclude by discussing the merits and deficiencies of our approach. 1
Using Links to Classify Wikipedia Pages.
Abstract. This paper contains a description of experiments for the 2008 INEX XML-mining track. Our goal for the XML-mining track is to explore whether we can use link information to improve classification accuracy. Our approach is to propagate category probabilities over linked pages. We find that using link information leads to marginal improvements over a baseline that uses a Naive Bayes model. For the initially misclassified pages, link information is either not available or contains too much noise. 1
Using parsimonious language models on web data.
In this paper we explore the use of parsimonious language models for web retrieval. These models are smaller thus more efficient than the standard language models and are therefore well suited for large-scale web retrieval. We have conducted experiments on four TREC topic sets, and found that the parsimonious language model results in improvement of retrieval effectiveness over the standard language model for all data-sets and measures. In all cases the improvement is significant, and more substantial than in earlier experiments on newspaper/newswire data.
A study of query length.
We analyse query length, and fit power-law and Poisson distributions to four different query sets. We provide a practical model for query length, based on the truncation of a Poisson distribution for short queries and a power-law distribution for longer queries, that better fits real query length distributions than earlier proposals.
Locating relevant text within XML documents.
Traditional document retrieval has shown to be a competitive approach in XML element retrieval, which is counter-intuitive since the element retrieval task requests all and only relevant document parts to be retrieved. This paper conducts a comparative analysis of document and element retrieval, highlights the relative strengths and weaknesses of both approaches, and explains the relative effectiveness of document retrieval approaches at element retrieval tasks.
The Impact of Positive%2C Negative and Topical Relevance Feedback.
Abstract This document contains a description of experiments for the 2008 Relevance Feedback track. We experiment with different amounts of feedback, including negative relevance feedback. Feedback is implemented using massive weighted query expansion. Parsimonious query expansion using only relevant documents and Jelinek-Mercer smoothing performs best on this relevance feedback track dataset. Additional blind feedback gives better results, except when the blind feedback set is of the same size as the explicit feedback set. On a small number of topics topical feedback is applied, which turns out to be mainly beneficial for early precision. 1
Where to Stop Reading a Ranked List%3F
Abstract: We document our participation in the TREC 2008 Legal Track. This year we focused solely on selecting rank cut-offs for optimizing the given evaluation measure per topic. 1
Report on the SIGIR 2008 workshop on focused retrieval.
Singapore. Nine paper were presented in three sessions and in a fourth session—joint with
Overview of the INEX 2007 Ad Hoc Track.
Abstract. This paper gives an overview of the INEX 2007 Ad Hoc Track. The main purpose of the Ad Hoc Track was to investigate the value of the internal document structure (as provided by the XML markup) for retrieving relevant information. For this reason, the retrieval results were liberalized to arbitrary passages and measures were chosen to fairly compare systems retrieving elements, ranges of elements, and arbitrary passages. The INEX 2007 Ad Hoc Track featured three tasks: For the Focused Task a ranked-list of non-overlapping results (elements or passages) was needed. For the Relevant in Context Task non-overlapping results (elements or passages) were returned grouped by the article from which they came. For the Best in Context Task a single starting point (element start tag or passage start) for each article was needed. We discuss the results for the three tasks, examine the relative effectiveness of element and passage retrieval. This is examined in the context of content only (CO, or Keyword) search as well as content and structure (CAS, or structured) search. 1
INEX 2007 Evaluation Measures.
Abstract. This paper describes the official measures of retrieval effectiveness that are employed for the Ad Hoc Track at INEX 2007. Whereas in earlier years all, but only, XML elements could be retrieved, the result format has been liberalized to arbitrary passages. In response, the INEX 2007 measures are based on the amount of highlighted text retrieved, leading to natural extensions of the well-established measures of precision and recall. The following measures are defined: The Focused Task is evaluated by interpolated precision at 1 % recall (iP[0.01]) in terms of the highlighted text retrieved. The Relevant in Context Task is evaluated by mean average generalized precision (MAgP)wherethe generalized score per article is based on the retrieved highlighted text. The Best in Context Task is also evaluated by mean average generalized precision (MAgP) but here the generalized score per article is based on the distance to the assessor’s best-entry point. 1
Unified Access to Heterogeneous Data in Cultural Heritage.
This paper addresses the prototypical problem of a cultural heritage institution with the ambition to disclose all of its content in a single, unified system. Like other enterprises, these institutions have heterogeneous collections distributed over multiple legacy systems. Our approach is to turn the metadata retrieval problem into a free-text retrieval problem by an unconditional merging of the heterogeneous sub-collections and flattening of all metadata structures. We investigate the viability of the approach by an extensive case study of a large museum. Our main findings are as follows: First, by converting all digital content to text, and indexing it with a standard IR system, we can effectively build a unified system providing access to all data. Second, an initial empirical evaluation shows superior performance in comparison with the legacy systems currently in use by the institute. Therefore, our third and overall finding is that our approach is a viable option to give access to heterogeneous collections. 1
Using and Detecting Links in Wikipedia.
Abstract. In this paper, we document our efforts at INEX 2007 where we participated in the Ad Hoc Track, the Link the Wiki Track, and the Interactive Track that continued from INEX 2006. Our main aims at INEX 2007 were the following. For the Ad Hoc Track, we investigated the effectiveness of incorporating link evidence into the model, and of a CAS filtering method exploiting the structural hints in the INEX topics. For the Link the Wiki Track, we investigated the relative effectiveness of link detection based on retrieving similar documents with the Vector Space Model, and then filter with the names of Wikipedia articles to establish a link. For the Interactive Track, we took part in the interactive experiment comparing an element retrieval system with a passage retrieval system. The main results are the following. For the Ad Hoc Track, we see that link priors improve most of our runs for the Relevant in Context and Best in Context Tasks, and that CAS pool filtering is effective for the Relevant in Context and Best in Context Tasks. For the Link the Wiki Track, the results show that detecting links with name matching works relatively well, though links were generally under-generated, which hurt the performance. For the Interactive Track, our test-persons showed a weak preference for the element retrieval system over the passage retrieval system. 1
Unified Access to Heterogeneous Data in Cultural Heritage.
This paper addresses the prototypical problem of a cultural heritage institution with the ambition to disclose all of its content in a single, unified system. Like other enterprises, these institutions have heterogeneous collections distributed over multiple legacy systems. Our approach is to turn the metadata retrieval problem into a free-text retrieval problem by an unconditional merging of the heterogeneous sub-collections and flattening of all metadata structures. We investigate the viability of the approach by an extensive case study of a large museum. Our main findings are as follows: First, by converting all digital content to text, and indexing it with a standard IR system, we can effectively build a unified system providing access to all data. Second, an initial empirical evaluation shows superior performance in comparison with the legacy systems currently in use by the institute. Therefore, our third and overall finding is that our approach is a viable option to give access to heterogeneous collections. 1
Where to start reading a textual XML document%3F
In structured information retrieval, the aim is to exploit document structure to retrieve relevant components, allowing the user to go straight to the relevant material. This paper looks at the so-called best entry points (BEPs), which are intended to give the user the best starting point to access the relevant information in the document. We examine the relationship between BEPs and relevant components in the INEX 2006 ad hoc assessments. Our main findings are the following: First, although documents are short, assessors often choose the best entry point some distance from the start of the document. Second, many of the best entry points coincide with the first relevant character in relevant documents, showing a strong relation between the BEP and relevant text. Third, we find browsing BEPs in articles with a single relevant passages, and container BEPs or context BEPs in articles with more relevant passages.
Evaluating relevant in context document retrieval with a twist.

Access to Legal Documents Exact Match%2C Best Match%2C and Combinations.
Abstract: In this paper, we document our efforts in participating to the TREC 2007 Legal track. We had multiple aims: First, to experiment with using different query formulations, trying to exploit the verbose topic statements. Second, to analyse how ranked retrieval methods can be fruitfully combined with traditional Boolean queries. Our main findings can be summarized as follows: First, we got mixed results trying to combine the original search request with terms extracted from the verbose topic statement. Second, by combining the Boolean reference run wit our ranked retrieval run allows us to get the high recall of the Boolean retrieval, whilst precision scores show an improvement over both the Boolean and the ranked retrieval runs. Third, we found out that if we treat the Boolean query as free text with varying degrees of interpretation of the original operator, we get competitive results. Moreover, both types of queries seem to capture different relevant documents, and the combination between the request text and the Boolean query leads to substantial gain in precision and recall. 1
Parsimonious Language Models for a Terabyte of Text.

Report on the SIGIR 2007 workshop on focused retrieval.
Report on
Overview of WebCLEF 2006.
Abstract. We report on the CLEF 2006 WebCLEF track devoted to crosslingual web retrieval. We provide details about the retrieval tasks, the used topic set, and the results of the participants. WebCLEF 2006 used a stream of known-item topics consisting of: (i) manual topics (including a selection of WebCLEF 2005 topics, and a set of new topics) and (ii) automatically generated topics (generated using two techniques). The results over all topics show that current CLIR systems are very effective, retrieving on average the target page in the top ranks. Manually constructed topics result in higher performance than and automatically generated ones. And finally, the resulting scores on automatic topics provide a reasonable ranking of the systems, showing that automatically generated topics are an attractive alternative in situations where manual topics are not readily available. 1
A Cross%2DLanguage Approach to Historic Document Retrieval.
Introduction Our cultural heritage, as preserved in libraries, archives and museums, is made up of documents written many centuries ago. Large-scale digitization initiatives, like DigiCULT [2], make these documents available to non-expert users through digital libraries and vertical search engines. For a user, querying a historic document collection may be a disappointing experience. Natural languages evolve over time, changing in pronunciation and spelling, and new words are introduced continuously, while older words
INEX 2006 Evaluation Measures.
Abstract. This paper describes the official measures of retrieval effectiveness employed at the ad hoc track of INEX 2006. 1
A Cross%2DLanguage Approach to Historic Document Retrieval.
Introduction Our cultural heritage, as preserved in libraries, archives and museums, is made up of documents written many centuries ago. Large-scale digitization initiatives, like DigiCULT [2], make these documents available to non-expert users through digital libraries and vertical search engines. For a user, querying a historic document collection may be a disappointing experience. Natural languages evolve over time, changing in pronunciation and spelling, and new words are introduced continuously, while older words
Filtering and Clustering XML Retrieval Results.
Abstract. As part of the INEX 2006 Adhoc Track, we conducted a range of experiments with filtering and clustering XML element retrieval results. Our basic retrieval engine retrieves arbitrary elements from the collection (corresponding to the Thorough Task). These runs are filtered to remove textual overlap between elements (corresponding to the Focused Task). The resulting runs can be clustered per article (corresponding to the All in Context Task). Finally, we select the “best ” element for each article (corresponding to the Best in Context Task). Our main findings are the following. First, a complete element index outperforms a restricted index based on section-structure, albeit the differences are small. Second, grouping non-overlapping elements per article does not lead to performance degradation, but may improve scores. Third, all restrictions of the “pure ” element runs (by removing overlap, by grouping elements per article, or by selecting a single element per article) lead to some but only moderate loss of precision. 1
INEX 2006 Evaluation Measures.
Abstract. This paper describes the official measures of retrieval effectiveness employed at the ad hoc track of INEX 2006. 1
Experiments with Document and Query Representations for a Terabyte of Text.
track, we conducted a range of experiments investigating the effects of larger test collections for both Adhoc and known-item topics. First, we looked at the amount of smoothing required for large-scale collections, and found that the largescale collections require little smoothing. Second, we investigated the relative effectiveness of various web-centric document representations based on document-text, incoming anchor-texts, and page titles. We found that these are of little value for the Adhoc task, but can provide crucial additional retrieval cues for the Named page finding task. Third, we studied the relative effectiveness of various query representations, both short and verbose statements of the topic of request, plus an intermediate query based on the most characteristic terms in the whole topic statement. We we found that using a more verbose query leads to an improvement of retrieval effectiveness. 1
Articulating information needs in XML query languages.
Document-centric XML is a mixture of text and structure. With the increased availability of document-centric XML documents comes a need for query facilities in which both structural constraints and constraints on the content of the documents can be expressed. How does the expressiveness of languages for querying XML documents help users to express their information needs? We address this question from both an experimental and a theoretical point of view. Our experimental analysis compares a structure-ignorant with a structure-aware retrieval approach using the test suite of the INEX XML retrieval evaluation initiative. Theoretically, we create two mathematical models of users ’ knowledge of a set of documents and define query languages which exactly fit these models. One of these languages corresponds to an XML version of fielded search, the other to the INEX query language. Our main experimental findings are: First, while structure is used in varying degrees of complexity, two thirds of the queries can be expressed in a fielded-search like format which does not use the hierarchical structure of the documents. Second, three quarters of the queries use constraints on the context of the elements to be returned; these contextual constraints cannot be captured by ordinary keyword queries. Third, structure is used as a search hint, and not as a strict requirement, when judged against the underlying information need. Fourth, the use of structure in queries functions as a precision enhancing device.
Web%2Dcentric language models.
We investigates language models for informational and navigational web search. Retrieval on the web is a task that differs substantially from ordinary ad hoc retrieval. We perform an analysis of prior probability of relevance for a wide range of non-content features, shedding further light on the importance of non-content features for web retrieval. Language models can naturally incorporate multiple document representations, as well as non-content information. For the former, we employ mixture language models based on document full-text, incoming anchor-text, and document titles. For the latter, we study a range of priors based on document length, URL structure, and link topology. We look at three types of topics—distillation, home page, and named page— as well as for a mixed query set. We find that the mixture models lead to considerable improvement of retrieval effectiveness for all topic types. The web-centric priors generally lead to further improvement of retrieval effectiveness.
Structured queries in XML retrieval.
marked up with XML tags. The tags add lightweight semantics to the text. Querying such collections calls for a hybrid query language: the text-rich nature of the documents suggest a content-oriented (IR) approach, while the mark-up allows users to add structural constraints to their IR queries. We propose an approach to such hybrid contentand -structure queries that decomposes a query into multiple content-only queries whose results are then combined in ways determined by the structural constraints of the original query. We report on ongoing work and present preliminary evaluation results, based on the INEX 2003 test set.
Overview of WebCLEF 2005.

EuroGOV Engineering a Multilingual Web Corpus.
Abstract. EuroGOV is a multilingual web corpus that was created to serve as the document collection for WebCLEF, the CLEF 2005 web retrieval task. EuroGOV is a collection of web pages crawled from the European Union portal, European Union member state governmental web sites, and Russian governmental web sites. The corpus contains over 3 million documents written in more than 20 different European languages. In this paper we provide a detailed description of the EuroGOV collection. 1
Combination Methods for Crosslingual Web Retrieval.
Abstract. We investigate a range of crosslingual web retrieval tasks using the test suite of the CLEF 2005 WebCLEF track, which features a stream of known-item topics in various languages. Our main findings are: (i) straightforward indexing and retrieval is effective for mixed monolingual web retrieval; (ii) standard machine translation methods are effective for bilingual web retrieval; but (iii) standard combination methods are ineffective for multilingual web retrieval; we analyze the failure and suggest an alternative Z-score normalization that leads to effective multilingual retrieval results. 1
The Effect of Structured Queries and Selective Indexing on XML Retrieval.
the INEX 2005 ad hoc track, covering the Thorough, Focused, and Fetch-Browse tasks and their structured (+S) counterparts. Our research questions for this round of INEX were threefold. Our first and main research question was to investigate the contribution of structural constraints to improved retrieval performance. Our main results were that the two types of structural constraints have different effects. Constraining the target of result elements gives improvements in terms of early precision. Constraining the context of result elements improves mean average precision. Our second research question was to experiment with selective indexing strategies based on either the length of elements, the tag-name of elements considered relevant in earlier INEX years, or simply by indexing all sections or articles. Our experiments show that disregarding 80–90% of the total number of elements does not decrease retrieval performance. Third, we considered the automatic creation of structured queries using blind feedback. Here, our results are inconclusive, mainly due to few queries used and lack of comparison to traditional blind feedback. 1
What Do Users Think of an XML Element Retrieval System%3F
the INEX 2005 Interactive Track, mainly focusing on a comparative experiment, in which the baseline system Daffodil/HyREX is compared to a home-grown XML element retrieval system (xmlfind). The xmlfind system provides an interface for an XML information retrieval search engine, using an index that contains all the individual XML elements in the IEEE collection. Our main findings are the following. First, test persons show appreciation for both systems, but xmlfind receives higher scores than Daffodil. Second, the interface seems to take the structural dependencies between retrieved elements into account in an appropriate way: although retrieved elements may be overlapping in whole or in part, none of the test persons regarded this as problematic. Third, the general opinion of the test persons on the usefulness of XML retrieval systems was unequivocally positive, and their responses highlight many of the hoped advantages of an XML retrieval system. 1
Biomedical Retrieval How Can a Thesaurus Help%3F.
Abstract. Searching specialized collections, such as biomedical literature, typically requires intimate knowledge of a specialized terminology. Hence, it can be a disappointing experience: not knowing the right terms to use and being unaware of synonyms or variations in terminology might result in low recall scores. We study the role of a thesaurus in the biomedical information retrieval process. We start by giving a description of vocabulary mismatch problems between natural language queries and relevant documents in biomedical literature search; we provide a detailed case study and observe the impact of vocabulary mismatch problems on retrieval effectiveness. Additionally, we analyze the associated MeSH thesaurus terms used to index the documents in the collection. Based on our observations, we propose a method for exploiting the MeSH thesaurus to improve retrieval effectiveness and, more specifically, to increase recall. We carry out a series of thesaurus-based retrieval experiments that show substantial performance improvements. We conclude with a detailed analysis of the retrieval results. 1
Combining Thesauri%2DBased Methods for Biomedical Retrieval.
Abstract: This paper describes our participation
Effective Smoothing for a Terabyte of Text.
track, we conducted a range of experiments investigating the effects of larger collections. Our main findings can be summarized as follows. First, we tested whether our retrieval system scales up to terabyte-scale collections. We found that our retrieval system can handle 25 million documents, although in terms of indexing time we are approaching the limits of a non-distributed retrieval system. Second, we hoped to find out whether results from earlier Web Tracks carry over to this task. For known-item search we found that, on the one hand, indegree and URL priors did not promote retrieval effectiveness, but that, on the other hand, the combination of different document representations improved retrieval effectiveness. Third, we investigated the role of smoothing for collections of this size. We found that larger collections require far less smoothing, especially for the adhoc task using very little smoothing leads to substantial gains in retrieval effectiveness. 1
The Importance of Length Normalization for XML Retrieval.

Processing content%2Doriented XPath queries.
Document-centric XML collections contain text-rich documents, marked up with XML tags that add lightweight semantics to the text. Querying such collections calls for a hybrid query language: the text-rich nature of the documents suggests a content-oriented (IR) approach, while the mark-up allows users to add structural constraints to their IR queries. Hybrid queries tend to be more expressive, which should lead—in principle—to better retrieval performance. In practice, the processing of these hybrid queries within an IR systems turns out to be far from trivial, because a delicate balance between structural and content information needs to be sought. We propose an approach to processing such hybrid content-and-structure queries that decomposes a query into multiple content-only queries whose results are then combined in ways determined by the structural constraints of the original query. We evaluate our methods using the INEX 2003 test-suite, and show (1) that effective ways of processing of content-oriented XPath queries are non-trivial, (2) that there are differences in the effectiveness for different topics types, but (3) that with appropriate processing methods retrieval effectiveness can improve.
Effective Translation%2C Tokenization and Combination for Cross%2DLingual Retrieval.
 Our approach to cross-lingual document retrieval starts from the assumption that effective monolingual retrieval is at the core of any cross-language retrieval system. We devote particular attention to three crucial ingredients of our approach to cross-lingual retrieval. First, effective tokenization techniques are essential to cope with morphological variations common in many European languages. Second, effective combination methods allow us to combine the best of different strategies. Finally, effective translation methods for translating queries or documents turn a monolingual retrieval system into a cross-lingual retrieval system proper. The viability of our approach is shown by a series of experiments in monolingual, bilingual, and multilingual retrieval.  
Improving Retrieval Effectiveness by Reranking Documents Based on Controlled Vocabulary.
Abstract. There is a common availability of classification terms in online text collections and digital libraries, such as manually assigned keywords or key-phrases from a controlled vocabulary in scientific collections. Our goal is to explore the use of additional classification information for improving retrieval effectiveness. Earlier research explored the effect of adding classification terms to user queries, leading to little or no improvement. We explore a new feedback technique that reranks the set of initially retrieved documents based on the controlled vocabulary terms assigned to the documents. Since we do not want to rely on the availability of special dictionaries or thesauri, we compute the meaning of controlled vocabulary terms based on their occurrence in the collection. Our reranking strategy significantly improves retrieval effectiveness in domain-specific collections. Experimental evaluation is done on the German GIRT and French Amaryllis collections, using the test-suite of the Cross-Language Evaluation Forum (CLEF). 1
Mixture Models%2C Overlap%2C and Structural Hints in XML Element Retrieval.
... Institute of the University of Amsterdam. We completely revamped our XML retrieval system, now implemented as a mixture language model on top of a standard search engine. To speed up structural reasoning, we indexed the collection’s structure in a separate database. Our main findings are as follows. First, we show that blind feedback improves retrieval effectiveness, but increases overlap. Second, we see that removing overlap from the result set decreases retrieval effectiveness for all metrics except the XML cumulative gain measure. Third, we show that ignoring the structural constraints gives good results if measured in terms of mean average precision; the structural constraints are, however, useful for achieving high initial precision. Finally, we provide a detailed analysis of the characteristics of one of our runs. Based on this analysis we argue that a more explicit definition of the INEX retrieval tasks is needed.  
The effectiveness of combining information retrieval strategies for European languages.
Building an effective Information Retrieval system requires various design choices, ranging from the weighting scheme to the type of morphological normalization. The combination of runs has become a standard technique to reap the benefits of different run types. Until now, systematic studies of the effectiveness of combination strategies have only been carried out for English. This paper provides an exploratory overview of the effectiveness of combination methods in nine European languages. We demonstrate that the combination of effective information retrieval strategies can lead to significant improvements of retrieval effectiveness. Furthermore, we analyze the relative impact of retrieving more relevant documents and of improved ranking of relevant documents. The experimental evidence is obtained using the 2003 testsuite of the cross-language evaluation forum (CLEF).  
Multiple sources of evidence for XML retrieval.
Document-centric XML collections contain text-rich documents, marked up with XML tags. The tags add lightweight semantics to the text. Querying such collections calls for a hybrid query language: the text-rich nature of the documents suggest a content-oriented (IR) approach, while the mark-up allows users to add structural constraints to their IR queries. We will show how evidence for relevancy from di#erent sources helps to answer such hybrid queries. We evaluate our methods using the INEX 2003 test set, and show that structural hints in hybrid queries help to improve retrieval e#ectiveness.
Length normalization in XML retrieval.
XML retrieval is a departure from standard document retrieval in which each individual XML element, ranging from italicized words or phrases to full blown articles, is a potentially retrievable unit. The distribution of XML element lengths is unlike what we usually observe in standard document collections, prompting us to revisit the issue of document length normalization. We perform a comparative analysis of arbitrary elements versus relevant elements, and show the importance of length as a parameter for XML retrieval. Within the language modeling framework, we investigate a range of techniques that deal with length either directly or indirectly. We observe a length bias introduced by the amount of smoothing, and show the importance of extreme length priors for XML retrieval. We also show that simply removing shorter elements from the index (by introducing a cut-o# value) does not create an appropriate document length normalization. Even after increasing the minimal size of XML elements occurring in the index, the importance of an extreme length bias remains.
Processing Content%2DAnd%2DStructure Queries for XML Retrieval.
marked up with XML tags. The tags add lightweight semantics to the text. Querying such collections calls for a hybrid query language: the text-rich nature of the documents suggest a content-oriented (IR) approach, while the mark-up allows users to add structural constraints to their IR queries. We propose an approach to such hybrid contentand -structure queries that decomposes a query into multiple content-only queries whose results are then combined in ways determined by the structural constraints of the original query. We report on ongoing work and present preliminary evaluation results, based on the INEX 2003 test set.
Language Models for Searching in Web Corpora.
 We describe our participation in the
Best%2DMatch Querying from Document%2DCentric XML.
On the Web, there is a pervasive use of XML to give lightweight semantics to textual collections. Such documentcentric XML collections require a query language that can gracefully handle structural constraints as well as constraints on the free text of the documents. Our main contributions are three-fold. First, we outline two fragments of XPath tailored to users that have varying degrees of understanding of the XML structure used, and give both syntactic and semantic characterizations of these fragments. Second, we extend XPath with an about function having a best-match semantics based on the relevance of the document component for the expressed information need. Third, we evaluate the resulting query language using the INEX 2003 test suite, and show that best-match approaches outperform exact-match approaches for evaluating content-and-structure queries.
Monolingual Document Retrieval for European Languages.
Recent years have witnessed considerable advances in information retrieval for European languages other than English. We give an overview of commonly used techniques and we analyze them with respect to their impact on retrieval e#ectiveness. The techniques considered range from linguistically motivated techniques, such as morphological normalization and compound splitting, to knowledge-free approaches, such as n-gram indexing. Evaluations are carried out against data from the CLEF campaign, covering eight European languages. Our results show that for many of these languages a modicum of linguistic techniques may lead to improvements in retrieval e#ectiveness, as can the use of language independent techniques.
Language%2DDependent and Language%2DIndependent Approaches to Cross%2DLingual Text Retrieval.
We investigates the e#ectiveness of language-dependent approaches  to document retrieval, such as stemming and decompounding,  and constrast them with language-independent approaches, such as character  n-gramming. In order to reap the benefits of more than one type of  approach, we also consider the e#ectiveness of the combination of both  types of approaches. We focus on document retrieval in nine European  languages: Dutch, English, Finnish, French, German, Italian, Russian,  Spanish, and Swedish. We look at four di#erent cross-lingual information  retrieval tasks: monolingual, bilingual, multilingual, and domain-specific  retrieval. The experimental evidence is obtained using the 2003 test suite  of the cross-language evaluation forum (CLEF).
XML retrieval what to retrieve%3F
The fundamental difference between standard information retrieval and XML retrieval is the unit of retrieval. In traditional IR, the unit of retrieval is fixed: it is the complete document. In XML retrieval, every XML element in a document is a retrievable unit. This makes XML retrieval more difficult: besides being relevant, a retrieved unit should be neither too large nor too small. The research presented here, a comparative analysis of two approaches to XML retrieval, aims to shed light on which XML elements should be retrieved. The experimental evaluation uses data from the Initiative for the Evaluation of XML retrieval (INEX 2002).
Approaches to Robust and Web Retrieval.
We describe our participation in the  TREC 2003 Robust and Web tracks. For the  Robust track, we experimented with the impact  of stemming and feedback on the worst scoring  topics. Our main finding is the effectiveness of  stemming on poorly performing topics, which  sheds new light on the role of morphological  normalization in information retrieval. For both  the home/named page finding and topic distillation  tasks of the Web track, we experimented  with different document representations and retrieval  models. Our main finding is effectiveness  of the anchor text index for both tasks, suggesting  that compact document representations  are a fruitful strategy for scaling-up retrieval  systems.
Combining Evidence for Cross%2DLanguage Information Retrieval.
This paper describes the official runs of our team for CLEF 2002. We  took part in the monolingual tasks for each of the seven non-English languages for  which CLEF provides document collections (Dutch, Finnish, French, German,  Italian, Spanish, and Swedish). We also conducted our first experiments for the  bilingual task (English to Dutch, and English to German), and took part in the  GIRT and Amaryllis tasks. Finally, we experimented with the combination of  runs.
Situation Calculus as Hybrid Logic First Steps.
Abstract. The situation calculus, originally conceived by John Mc-Carthy, is one of the main representation languages in artificial intelligence. The original papers introducing the situation calculus also highlight the connection between the fields of artificial intelligence and philosophical logic (especially modal logics of belief, knowledge, and tense). Modal logic changed enormously since the 60s. This paper sets out to revive the connection between situation calculus and modal logic. In particular, we will show that quantified hybrid logic, QHL, is able to express situation calculus formulas often more natural and concise than the original formulations. The main contribution of this paper is a new quantified hybrid logic with temporal operators and action modalities, tailor-made for expressing the fluents of situation calculus.
The Importance of Morphological Normalization for XML Retrieval.
Current information retrieval systems typically ignore structural aspects of documents, solely focusing on the textual content instead. But documents containing additional structure in the form of HTML, XML, or SGML mark-up are pervasive on the Internet. The XML retrieval task presents a number of challenges for information retrieval, for we can no longer rely on the appropriate unit of retrieval to be fixed, or to be known beforehand. This implies that the effectiveness of standard IR techniques, such as morphological normalization methods, may not carry over to this particular task. This paper describes the fully automatic runs for the INEX 2002 task submitted by the Language and Inference Technology Group at the University of Amsterdam. We investigate the effectiveness of two standard approaches to morphological normalization, both a linguistically motivated stemming algorithm and a knowledge-poor character n-gramming technique. Our results show that morphological normalization is an important issue for XML retrieval. For all measurements, the combined run and the n-gram run perform better than the stemmed run. 1
Notions of Indistinguishability for Semantic Web Languages.
Abstract. The paper reviews the notions of expressiveness of description logics from (N. Kurtonina and M. de Rijke. Expressiveness of concept expressions in first-order description logics. Artificial Intelligence, 107:303–333, 1999) and exemplifies their use in the development in Semantic Web languages. The notion of bisimulation—which characterizes the description logic ALC—provides a direct link to what’s in the field of sociology called social network analysis. The perspective on data in this field—data are represented as labeled graphs—fits exactly the modeling intuitions of web languages like oil and daml+oil. This is exemplified in the study of trophic networks. A further connection is established between web languages and hybrid logic, and an extension of oil with a limited form of self reference is proposed. 1
The University of Amsterdam at TREC 2002.
We describe our participation in the  TREC 2002 Novelty, Question answering, and Web  tracks. We provide a detailed account of the ideas  underlying our approaches to these tasks. All our  runs used the FlexIR information retrieval system.
Situation Calculus as Hybrid Logic First Steps.
Abstract. The situation calculus, originally conceived by John Mc-Carthy, is one of the main representation languages in artificial intelligence. The original papers introducing the situation calculus also highlight the connection between the fields of artificial intelligence and philosophical logic (especially modal logics of belief, knowledge, and tense). Modal logic changed enormously since the 60s. This paper sets out to revive the connection between situation calculus and modal logic. In particular, we will show that quantified hybrid logic, QHL, is able to express situation calculus formulas often more natural and concise than the original formulations. The main contribution of this paper is a new quantified hybrid logic with temporal operators and action modalities, tailor-made for expressing the fluents of situation calculus.
On Criteria for Formal Theory Building Applying Logic and Automated Reasoning Tools to the Social Sciences.
This paper provides practical operationalizations of criteria for evaluating scientific theories, such as the consistency and falsifiability of theories and the soundness of inferences, that take into account definitions. The precise formulation of these criteria is tailored to the use of automated theorem provers and automated model generators—generic tools from the field of automated reasoning. The use of these criteria is illustrated by applying them to a first order logic representation of a classic organization theory, Thompson’s Organizations in Action.
Formal Theory Building Using Automated Reasoning Tools.
The merits of representing scientific theories in formal logic are well-known. Expressing a scientific theory in formal logic explicates the theory as a whole, and the logic provides formal criteria for evaluating the theory, such as soundness and consistency. On the one hand, these criteria correspond to natural questions to be asked about the theory: is the theory contradiction-free? (is the theory logically consistent?) is the theoretical argumentation valid? (can a theorem be soundly derived from the premises?) and other such questions. On the other hand, testing for these criteria amounts to making many specific proof attempts or model searches: respectively, does the theory have a model? can we find a proof of a particular theorem? As a result, testing for these criteria quickly defies manual processing. Fortunately, automated reasoning provides some valuable tools for this endeavor. This paper discusses the use of first-order logic and existing automated rea...
